Fix CUDA arch selection and provide a portable arch list - #280

Problem

All CUDA builds based on the CMakeLists.txt that do not explicitly set CMAKE_CUDA_ARCHITECTURES are only built for nvcc's default architecture: sm_75 on CUDA 13, sm_52 on CUDA 12.

The fallback to set CMAKE_CUDA_ARCHITECTURES=native is never reached (and in addition is only supported on CMake >= 3.24).

=> For local builds this means: built for sm_75 (CUDA 13) / sm_52 (CUDA 12), even on an RTX 5090 (sm_120).
=> For Docker builds this also means: no portable set of architectures is built, only the minimal sm_52 or sm_75.

The Windows prebuilts are not affected (they pass a curated arch list); build_windows.ps1 auto explicitly targets the local GPU when one is present.

Root cause

We set cmake_minimum_required(VERSION 3.20) which activates policy CMP0104: during enable_language(CUDA), CMake auto-initializes CMAKE_CUDA_ARCHITECTURES from nvcc's compile default whenever the user doesn't set it. The seed comes from the compiler-id probe - it does not query the local GPU. It is sm_75 on CUDA 13, sm_52 on CUDA 12.

Example (CUDA 12):

• On an RTX 5090 we see:
  -- Using CMAKE_CUDA_ARCHITECTURES=52 CMAKE_CUDA_ARCHITECTURES_NATIVE=120a-real
• On a host without GPU (e.g. GitHub runner)
  -- Using CMAKE_CUDA_ARCHITECTURES=52 CMAKE_CUDA_ARCHITECTURES_NATIVE=No CUDA devices found.-real

Since CMAKE_CUDA_ARCHITECTURES is initialized by CMake, the fallback to "native" is never used:

if (CMAKE_CUDA_ARCHITECTURES)
    set_target_properties(engine_runtime PROPERTIES CUDA_ARCHITECTURES "${CMAKE_CUDA_ARCHITECTURES}")
else()
    set_target_properties(engine_runtime PROPERTIES CUDA_ARCHITECTURES native)
endif()

Solution

This PR brings the implementation closer to llama.cpp:

• We define a set of architectures to include in a default build. This is applied automatically if CMAKE_CUDA_ARCHITECTURES is not set.
  • Defaults are:
    • 50/61/70-virtual, if < 13.0
    • + 75/80-virtual
    • + 86-real;
    • + 89-real 90-virtual, if ≥ 11.8
    • + 120a-real, if ≥ 12.8
    • + 121a-real, if ≥ 12.9
• Users can reduce build times by explicitly setting CMAKE_CUDA_ARCHITECTURES=native (CMake >= 3.24) or by defining a custom set, e.g. CMAKE_CUDA_ARCHITECTURES=120a-real

For Docker builds we do not set CMAKE_CUDA_ARCHITECTURES and thus build for the full defined arch set.

Details

• The default list is set before enable_language(CUDA) in CMakeLists.txt. That is the only point where "the user didn't set anything" is still detectable, because the CMake seed happens inside that call. The old if/else fallback quoted above sat after the seed and was therefore dead code - it is removed, since CMAKE_CUDA_ARCHITECTURES automatically initializes the CUDA_ARCHITECTURES property of every CUDA target.
• Suffixes: -virtual = PTX (compiled on the fly by the driver, forward-compatible), -real = SASS (native machine code).
• The default list is identical to llama.cpp's ggml-cuda default.
• Plain 12X values (e.g. 120, or native resolved on a Blackwell GPU) are upgraded to 12Xa - the a variant unlocks Blackwell's FP4 tensor cores. Each replacement is logged.
• native requires CMake >= 3.24. On CMake 3.20-3.23 it falls back to the default list with a status message (same gate as llama.cpp's GGML_NATIVE, but visible instead of silently ignored).
• The configure log prints the final list (-- Using CMAKE_CUDA_ARCHITECTURES=...), so the result is easy to verify.
• The CUDAARCHS environment variable keeps working (CMake reads it before the seed).
• .devops/cuda.Dockerfile contains no arch list at all - CMakeLists.txt is the single source of truth. Custom images can still override via --build-arg CUDA_DOCKER_ARCH="89-real;...".

Consequences / Remarks

• Since we now define a default arch set, build times for default builds (without CMAKE_CUDA_ARCHITECTURES) increase by roughly 4.5x (CUDA 12 Docker build on a 24-core CPU).

Open issues / TODO

• Update docs
• Check workflows
• Follow up? Avoid having two separate curated arch lists in CMakeLists.txt vs windows build scripts